REMARKS 

1 . In paragraph 2 of the Official Action, the Examiner indicates that the Stephen B. Wicker 
document has not been considered. The Applicant notes that this document is an entire book 
and acknowledges that the Examiner has not taken it into consideration. 

2. In paragraph 4 of the Official Action, the Examiner suggests that a number of claimed 
features were not described in the specification in such a way as to reasonably convey to one 
skilled in the relevant art that the inventor, at the time the application was filed, had 
possession of the claimed invention. In reply, the Applicant submits that those features were 
disclosed in the specification as filed. 

Arguments in relation to each of the claimed features appear below: 

(a) "the map of the geographic area and the coded data being printed substantially 
simultaneously ": 

(i) In Figure 1 and on page 10, lines 6 to 1 1 of the specification, "netpages" are 
described as including both coded data and graphic data: 

"As illustrated in Figure 1, a printed netpage 1 can represent a 
interactive form which can be filled in by the user both physically, on 
the printed page, and "electronically", via communication between 
the pen and the netpage system. The example shows a "Request" form 
containing name and address fields and a submit button. The netpage 
consists of graphic data 2 printed using visible ink, and coded data 3 
printed as a collection of tags 4 using invisible ink. " 

(ii) On page 10, lines 22 to 26 of the specification, the netpage printer 601 is 
described as printing netpages: 

"The netpage printer 601, preferred forms of which are described in 
our earlier application USSN 09/575, 155 (docket no, NPP003US) 
and our co-filed application USSN 09/693,514 (docket no. 
NPS024US), is able to deliver, periodically or on demand, 
personalized newspapers, magazines, catalogs, brochures and other 
publications, all printed at high quality as interactive netpages. " 

Additional disclosure of the manner in which netpage printers encode 
and print the coded data contained in netpages, can be found in cross- 
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referenced United States patent application No. 09/575,109 
(PEC02US). 

(iii) Since "netpages " include both coded data and graphic data and since the 
netpage printer 601 is described as printing netpages, one of ordinary skill in the art 
would understand that the netpage printer is described as printing both the coded data 
and the graphic data substantially simultaneously. 

(iii) In Figure 14 and on page 24, lines 15 to 18 of the specification the netpage 
printer is described as printing netpages in which the "graphic data " takes the form 
of maps of countries, regions, cities and areas. 

"If the user selects a point on a map or globe and clicks the <Print 
Country Map> button, a map 524 of the selected country is printed. 
The user may also print a map 524 of the selected location (country, 
region, or city) by clicking the <Print Map> button on the 
Information page, as shown in Figure 14. " 

(iv) Since maps of countries, regions, cities and areas are maps of "geographic 
areas" and since the netpage printer is described as printing netpages containing 
such maps of geographic areas, one of ordinary skill in the art would understand that 
the method being described involves "the map of the geographic area and the coded 
data being printed substantially simultaneously. " 

(v) For these reasons, the Applicant submits that "the map of the geographic 
area and the coded data being printed substantially simultaneously" was in the 
specification as filed and asks that the Examiner withdraw this objection. 

(b) "generating the indicating data based at least partially on sensing at least some of 
the coded data in the vicinity of the position ": 

(i) Page 13, line 28 to page 14 line 3 of the specification describes the netpage 
pen as generating the page identifier and position of the pen relative to the page (the 
indicating data) by sensing the tags (coded data) even on a single click on the page 
(in the vicinity of the position of the sensing device): 

"A tag is sensed by an area image sensor in the netpage pen, and the 
tag data is transmitted to the netpage system via the nearest netpage 
printer. The pen is wireless and communicates with the netpage 
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printer via a short-range radio link Tags are sufficiently small and 
densely arranged that the pen can reliably image at least one tag even 
on a single click on the page. It is important that the pen recognize 
the page ID and position on every interaction with the page, since the 
interaction is stateless. " 

(ii) The netpage pen is therefore described as "generating the indicating data 
based at least partially on sensing at least some of the coded data in the vicinity of 
the position. " 

(iii) For this reason, the Applicant submits that "generating the indicating data 
based at least partially on sensing at least some of the coded data in the vicinity of 
the position" was in the specification as filed and asks that the Examiner withdraw 
this objection. 

(c) "the printer being adapted to print the map and the coded data substantially 
simultaneously ": 

(i) The arguments and specification extracts mentioned in paragraph (a) of this 
section apply equally to this claim feature. 

(d) "comprising a non-electronic printed surface displaying coded data indicative of a 
plurality of reference points of the globe ": 

(i) Page 20, lines 29 and 30 describe a globe printed as a netpage: 

"The surface of a globe (i.e. a sphere representing the earth) can also be 
printed as a netpage. " 

Further disclosure of netpage tags being applied to a spherical surface, such 
as a globe, can be found in cross-referenced United States patent application 
No. US 09/575,129 (NPT002) in the section entitled "8.1.2 Spherical Surface 
Tag Tiling" beginning at the end of page 74. 

(ii) Page 9, lines 25 to 27 describe netpages as being printed on ordinary paper: 

"In its preferred form, the netpage system relies on the production of and 
human interaction with, netpages. These are pages of text, graphics and 
images printed on ordinary paper or other media, but which work like 
interactive web pages. " 



(iii) "Ordinary paper " is a non-electronic surface. 

(iv) Since these passages disclose a globe printed as a netpage, and since a 
netpage is a non-electronic printed surface, these passages disclose a globe 
"comprising a non-electronic printed surface displaying coded data indicative of a 
plurality of reference points of the globe. " 

(v) For these reasons, the Applicant submits that a globe "comprising a non- 
electronic printed surface displaying coded data indicative of a plurality of reference 
points of the globe " was in the specification as filed and asks that the Examiner 
withdraw this objection. 

3. In order to more clearly distinguish the claimed invention from the cited art, the 
Applicant has sought to delete claims 1 to 5, 7 and 12 and to amend independent claims 6 
and 1 1 . The Applicant submits that independent claim 6 is novel and inventive in light of 
the cited art for the following reasons: 

(a) Neither Dymetman nor Conroy et al disclose a printer "adapted to print the map 

and the coded data substantially simultaneously. " In paragraph 6 of the Official Action the 
Examiner acknowledges that Conroy et al. "fails to specifically disclose printing a map, 
including coded data; the map of the geographic area and the coded data being printed 
substantially simultaneously. " The Examiner argues, however that these feature are 
disclosed on page 396, section 3 of the Dymetman article. The Applicant concedes that the 
map of Europe disclosed in the Dymetman article is printed with two layers of ink, one 
being the coded layer and the other being the conventional, visible layer. However, the 
Dymetman article does not disclose that these two layers are printed substantially 
simultaneously. In contrast, the following sentences from the Dymetman article teach that 
the coded layer of ink in the Dymetman arrangement is pre-printed by an authorised 
producer and the layer of conventional, visible ink is added later by a publisher: 

"These sheets are produced by publishers, who buy apparently blank sheets 
of Intelligent Paper from an authorized producer. The publishers can mark 
them with conventional visible inks in any way they choose. " (Page 394, lines 
1 to 3) 
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"This way of proceeding exploits the natural tendency of publishers to buy 
Intelligent Paper sheets in bulk, so that it may be known by the first router 
that a certain number of consecutive page-ids are "owned" by a certain 
publisher. " (Page 398, lines 4 to 7) 

In addition, as at the date of the invention, the state of printer technology was such 
that one of ordinary skill in the art would not have readily conceived of a printer capable of 
printing both layers of ink simultaneously, given the complex nature of the coded data layer. 
However, the Applicant's invention does disclose printing technology capable of such 
simultaneous printing. 

(b) Neither of the citations disclose a sensing device having the features defined in 
amended claim 6. The sensing device in the Conroy et al arrangement does not have an 
image sensor at all. In Dymetman, although there is some disclosure on how the "pointer" 
might be used, there is no disclosure of how it works. In fact, the text suggests that the 
"writer-pointer " had not even been developed at the time of writing. For example, on page 
404 under the heading "6 Conclusion " the Dymetman article reads: 

"We have discussed some of the potential applications of Intelligent Paper, and 
presented the underlying technology in its broad outline. Some aspects of this 
technology - invisible ink, writer-pointer — are not fully mature yet, but progress is 
being rapidly made. " 

There is certainly no disclosure in either Dymetman or Conroy of a sensing device 
which has a processor which is adapted to: 

"(i) identify at least some of the coded data from one or more of the 

captured images; 

(ii) determine an orientation, within the captured images, of at least some of 
the coded data; 

(Hi) decode at least some of the coded data; and 

(iv) generate, using at least some of the decoded coded data, indicating data 
indicative of the identity of the map and a position of the sensing device relative to 
the map " 

Claim 6 is therefore not anticipated by the Dymetman and Conroy citations. Similar 
comments apply in relation to claim 1 1 . 
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4. In anticipation of the examiner questioning where in the specification these features 
of the sensing device are disclosed, the Applicant makes the following comments: 

(a) On page 10, lines 17 to 21 of the specification as filed, the reader is referred to one of 
the Applicant's co-pending applications for further details on the sensing device: 

"As illustrated in Figure 2, the netpage pen 101, a preferred form of which is 
described in our earlier application USSN 09/575, 1 74 (docket no. NPS001 US), 
works in conjunction with a netpage printer 601, an Internet-connected printing 
appliance for home, office or mobile use. The pen is wireless and communicates 
securely with the netpage printer via a short-range radio link 9. " 

(b) In the Applicant's co-pending application USSN 09/575, 1 74, the Netpage pen is 
described in significant detail. In support of these claim amendments, the Examiner is 
particularly referred to the following sections of that document: 

(i) page 65, line 1 8 to page 66 line 1 1 under the heading "6.4 Pen Processing" and 

(ii) page 62 line 7 to page 64 line 14 under the heading "6.2 Pen Controller. " 

For the Examiner's ease of reference, pages 62 to 66 of USSN 09/575,174 are 
attached. 

5. For the above reasons the Applicant submits that amended claims 6 and 8 to 1 1 are 
novel and inventive in light of the cited art and that the claim amendments introduce no new 
matter. 
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CONCLUSION 

It is respectfully submitted that all of the Examiner's objections have been successfully 
traversed. Accordingly, it is submitted that the application is now in condition for 
allowance. Reconsideration and allowance of the application is courteously solicited. 

Very respectfully, 
Applicant: 




PAUL LAPSTUN 



JACQtJELDtfE ANNE LAPSTUN 



KIA SILVERBROOK 

C/o: Silverbrook Research Pty Ltd 

393 Darling Street 
Balmain NSW 2041, Australia 

Email : Kia.silverbrook@silverbrookresearch .com 

Telephone: +612 981 8 6633 
Facsimile: +61 2 9818 671 1 
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VERSION WITH MARKINGS TO SHOW CHANGES MADE 



CO-PENDING APPLICATIONS 

Various methods, systems and apparatus relating to the present invention are 

disclosed in the following co-pending applications filed by the applicant or assignee of the 
present invention on October 20, 2000: 




09/693,415- 
09/693,515- 
09/693 ,690 (NPA061US) , 
09/693,341 (NPS008US) , 
09/693,301 (NPPC1) , 
09/693 ,5 1 0 (UP03US) , 




09/693,280 (NPA010US) , 
09/693 ,647 (NPA059US) , 
09/693,216 (NPS00-1US) , 
09/696,5 14-(NPS034US), 
09/693,70 4 (UP02US) , 
09/693,335 (UP05US) 



09/693,219h 
09/693,705- 

09/693,593 (NPB006US) , 
09/696,473 (NPS013US) 
09/693,388 (UP01US) , 
09/09/693,336 (UP04US) , 
The disclosures of these co-pending applications are incorporated herein by reference. 

Various methods, systems and apparatus relating to the present invention are 
disclosed in the following co-pending applications filed by the applicant or assignee of the 
present invention on September 15, 2000: 

09/663,579 (NPA024US) , 09/669,599 (NPA025US) , 09/663,701 (NPA017US) , 
09/663,640 (NPA019US) , 

The disclosures of these co-pending applications are incorporated herein by reference. 

Various methods, systems and apparatus relating to the present invention are 
disclosed in the following co-pending applications filed by the applicant or assignee of the 
present invention on June 30, 2000: 

09/609,139 (NPA011US) , 09/608,970 (NPA015US) , 09/609,039 (NPA022US) , 
09/607,852 (NPA026US) , 09/607,656 (NPA038US) , 09/609,132 (NPA041US) , 
09/609,303 (NPA050US) , 09/610,095 (NPA051US) , 09/609,596- 
09/607,843 (NPA063US) , 
09/609,553 (NPA068US) , 
09/608,022 (NPA072US) , 

09/607,657 (NPB005US) , 09/608,920 (NPP019US) , 09/607,985- 
09/607,990 (PEC05US) , 09/607,196 (PEC06US) , 09/606,999 (PEC07US) 
The disclosures of these co-pending applications are incorporated herein by reference. 

Various methods, systems and apparatus relating to the present invention are 
disclosed in the following co-pending applications filed by the applicant or assignee of the 
present invention on 23 May 2000: 



09/607,605 (NPA065US) , 09/608, 1 78 (NPA067US) 
09/609,233 (NPA069US) , 09/609,149 (NPA071US) 
09/609,232 (NPB003US) , 09/607,84 4 (NPB004US) 
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09/575,197 (NPA001US) , 
09/575, 1 32 (NPA005US) , 
09/575,130 (NPA00 8 US) , 
09/575,118 (NPA012US) , 
09/575,14 1 (NPA018US) , 
09/575, 1 85 (NPA021US) , 
09/575,192 (NPA018US) , 
09/575,156 (NPB002US) , 
09/575,150 (NPK001US) , 
09/575,128 (NPM002US) , 
09/575,179 (NPN001US) , 
09/575,187 (NPP001US) , 
09/575,198 (NPP008US) , 
09/575,146 (NPP018US) , 
09/575,168 (NPS020US) , 
09/575,12 1 (NPT003US) , 
09/575,162 (NPX003US) , 
09/575,171 (NPX011US) , 
09/575,125 (IJM52US) , 
09/575,190 (MJ12US) , 
09/575,127 (MJ15US) , 
09/575,147 (MJ58US) , 
09/575,1 15 (PAK01US) , 
09/575 1 1 2 (PAK07US) , 
09/575,109 (PEC02US) , 
The disclosures of these co- 



09/575,195- 
09/575,123 (NPA006US) , 
09/575,165 (NPA009US) , 
09/575,131 (NPA016US) , 
09/575,139 (NPA019US) , 
09/575,191 (NPA030US) , 
09/575,181 (NPA075US) , 
09/575,183 (NPK002US) , 
09/575,1 69- 



09/575,159- 
09/575,148 (NPA007US) , 
09/575,153 (NPA010US) , 
09/575,1 16 (NPA017US) 

09/575,145 (NPA035US) , 
09/575,193 (NPB001US) , 
09/575,160 (NPK003US) , 
09/575,18 1 (NPM001US) , 



09/575,180 (NPM003US) , 09/575,149- 
09/575,133 (NPP005US) , 09/575,143 (NPP006US) , 



09/575,155 (NPP003US) , 
09/575 1 78 (NPP016US) , 
09/575, 1 7 1 (NPS001US) , 
09/575,15 1 (NPT001US) , 
09/575,188 (NPT001US) , 
09/575,172 (NPX00 8 US) , 
09/575,161 (NPX016US) , 
09/575,142 (MJ10US) , 
09/575,138 (MJ13US) , 
09/575,158 (MJ31US) , 
09/575,152 (MJ62US) , 
09/575,1 1 1 (PAK05US) , 
09/575,1 1 1 (PAK08US) , 




09/575, 1 96 (NPP007US) , 
09/575,16 1 (NPP017US) , 
09/575,163 (NPS003US) , 
09/575,129- 
09/575,189- 
09/575,170 (NPX011US) , 
09/575,141 (IJ52US) , 
09/575,140 (MJUUS) , 
09/575,126 (MJ11US) , 
09/575,117 (MJ17US) , 
09/575,176 (MJ63US) , 
09/575,113 (PAK06US) , 
09/575,108 (PECO 1 US) , 



09/575,1 10 (PEC03US) 
pending applications are incorporated herein by reference. 



In the Claims: 

Claims 6, 8, 9 and 11 have been amended as follows: 



6. (Twice Amended) A system for enabling a user to designate, in a computer system, 

at least one geographic location, the system including: 

a surface on which is disposed a map of a geographic area, the geographic area 
including the at least one geographic location, the map including coded data indicative of an 
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identity of the map and of a plurality of reference points of the map; 

a printer for printing the map, including the coded data onto the surface , ea 
demand, the printer being adapted to print the map and the coded data substantially 
simultaneously; and 

a sensing device comprising: 

(a) an image sensor adapted to capture images of at least some of the coded data 
when the sensing device is placed in an operative position relative to the surface; and 

(b) a processor adapted to: 

{i) identify at least some of the coded data from one or more of the 

captured images: 

(ii) determine an orientation, within the captured images, of at least 

some of the coded data: 

(iii) decode at least some of the coded data: and 

(iv) generate, using at least some of the decoded coded data, indicating 

data indicative of the identity of the map and a position of the sensing device 
relative to the map: and 

a computer system configured to for receiving receive the indicating data from ar-the 

sensing device operated by th e us e r, th e indicating data r e garding th e identity of the map and 
a position of th e s e nsing d e vic e r e lativ e to th e map, th e s e nsing d e vic e , wh e n placed in an 
op e rativ e position r e lative to the map, g e n e rating th e indicating data bas e d at l e ast partially 
on sensing at least some of th e cod e d data in th e vicinity of the position; 

wh e r e in th e comput e r syst e m is configur e d and to identify, from the indicating data, 

the at least one geographic location. 

8. (Amended) A system according to claim 7-6_wherein the map contains at least one 

of the following categories of map information: 

(a) geographic features of the geographic area; 

(b) cities in the geographic area; 

(c) countries related to the geographic area; 

(d) different views of the geographic area; 

(e) topography of the geographic area; 
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(f) vegetation of the geographic area; 

(g) average rainfall for the geographic area; 

(h) seasonal temperatures for the geographic area; and 

(i) population for the geographical area. 

9. (Amended) A system according to claim 6 or claim 7 further including a map 

control page including at least one printed map control; wherein the computer system is 
configured to perform an action associated with the map control when the map control is 
designated by the user using the sensing device. 

1 1 . (Twice Amended) A system for enabling a user to designate, in a computer system, 
at least one geographic location, the system including: 

a glob e having a surface on which is disposed a global map, the global map 
including the at least one geographic location and , th e glob e comprising a non el e ctronic 
printed surface displaying coded data indicative of a plurality of reference points of the 
globe; 

a sensing device comprising: 

(a) an image sensor adapted to capture images of at least some of the coded data 
when the sensing device is placed in an operative position relative to the globe: and 

(b) a processor adapted to: 

£i) identify at least some of the coded data from one or more of the 

captured images: 

£ii) determine an orientation, within the captured images, of at least 

some of the coded data: 

(iii) decode at least some of the coded data: and 

(iv) generate, using at least some of the decoded coded data, indicating 
data indicative of a position of the sensing device relative to the globe: and 

a computer system for r e c e ivin g configured to receive the indicating data from a-the 
sensing device op e rated by the us e r, th e indicating data r e garding a position of the sensing 
d e vice relativ e to the surfac e of the globe, th e s e nsing d e vic e , when plac e d in an operative 
position r e lativ e to the surface of th e glob e , g e n e ratin g and to th e indicating data based at 
l e ast partially on s e nsing at l e ast some of the cod e d data in th e vicinity of th e position; 



/Hk2M/^t/< To 

-62- 

119 or stylus nib 121. The IR photodiode 144 detects light from the IR LED 143 via 
reflectors (not shown) mounted on the slider blocks 123 and 124. 

Rubber grip pads 141 and 142 are provided towards the end 108 of the housing 
102 to assist gripping the pen 101, and top 105 also includes a clip 142 for clipping the 
5 pen 101 to a pocket. 

6.2 Pen Controller 

The pen 101 is arranged to determine the position of its nib (stylus nib 121 or 

ink cartridge nib 1 19) by imaging, in the infrared spectrum, an area of the surface in the 
vicinity of the nib. It records the location data from the nearest location tag, and is 
10 arranged to calculate the distance of the nib 121 or 119 from the location tab utilising 
optics 135 and controller chip 134. The controller chip 134 calculates the orientation of 
the pen and the nib-to-tag distance from the perspective distortion observed on the 
imaged tag. 

Control data from the location tag may include control bits instructing the 
15 pen 101 to activate its "active area" LED (this is in fact one mode of the tri-color LED 
116, which becomes yellow when the pen determines, from the control data, that the area 
that is being imaged is an "active area"). Thus, a region on the surface which 
corresponds to the active area of a button or hyperlink may be encoded to activate this 
LED, giving the user of the pen visual feedback that the button or hyperlink is active 
20 when the pen 101 passes over it. Control data may also instruct the pen 101 to capture 
continuous pen force readings. Thus a region on the surface which corresponds to a 
signature input area can be encoded to capture continuous pen 101 force. 

Pen 101 action relative to the surface may comprise a series of strokes. A 
stroke consists of a sequence of time-stamped pen 101 positions on the surface, initiated 

25 by pen-down event and completed by a subsequent pen-up event. Note that pen force can 
be interpreted relative to a threshold to indicate whether the pen is "up" or "down", as 
well as being interpreted as a continuous value, for example when the pen is capturing a 
signature. The sequence of captured strokes constitutes so-called "digital ink". Digital 
ink can be used with a computing system to form the basis for the digital exchange of 

30 drawings and handwriting, for on-line recognition of handwriting, and for on-line 
verification of signatures. 
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Utilising the RF chip 133 and antenna 1 12 the pen 101 can transmit the digital 
ink data (which is encrypted for security and packaged for efficient transmission) to the 
computing system. 

5 When the pen is in range of a receiver, the digital ink data is transmitted as it is 

formed. When the pen 101 moves out of range, digital ink data is buffered within the pen 
101 (the pen 101 circuitry includes a buffer arranged to store digital ink data for 
approximately 12 minutes of the pen motion on the surface) and can be transmitted later. 

The controller chip 134 is mounted on the second flex PCB 129 in the pen 101. 
1 0 Figure 1 0 is a block diagram illustrating in more detail the architecture of the controller 
chip 134. Figure 10 also shows representations of the RF chip 133, the image sensor 132, 
the tri-color status LED 1 16, the IR illumination LED 13 1, the IR force sensor LED 143, 
and the force sensor photodiode 144. 

The pen controller chip 134 includes a controlling processor 145. Bus 146 
15 enables the exchange of data between components of the controller chip 134. Flash 
memory 147 and a 512 KB DRAM 148 are also included. An analog-to-digital converter 
149 is arranged to convert the analog signal from the force sensor photodiode 144 to a 
digital signal. 

An image sensor interface 152 interfaces with the image sensor 132. A 
20 transceiver controller 153 and base band circuit 154 are also included to interface with 
the RF chip 133 which includes an RF circuit 155 and RF resonators and inductors 156 
'* connected to the antenna 112. 

The controlling processor 145 captures and decodes location data from tags 
from the surface via the image sensor 132, monitors the force sensor photodiode 144, 
25 controls the LEDs 116, 131 and 143, and handles short-range radio communication via 
the radio transceiver 153. It is a medium-performance (~40MHz) general-purpose RISC 
processor. 

The processor 145, digital transceiver components (transceiver controller 153 
and baseband circuit 154), image sensor interface 152, flash memory 147 and 512KB 
30 DRAM 148 are integrated in a single controller ASIC. Analog RF components (RF 
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circuit 155 and RF resonators and inductors 156) are provided in the separate RF chip. 

The image sensor is a 215x215 pixel CCD (such a sensor is produced by 
Matsushita Electronic Corporation, and is described in a paper by Itakura, K T 
Nobusada, N Okusenya, R Nagayoshi, and M Ozaki, "A 1mm 50k-Pixel IT CCD Image 
5 Sensor for Miniature Camera System", IEEE Transactions on Electronic Devices, Volt 
47, number 1, January 2000, which is incorporated herein by reference) with an IR filter. 

The controller ASIC 134 enters a quiescent state after a period of inactivity 
when the pen 101 is not in contact with a surface. It incorporates a dedicated circuit 150 
which monitors the force sensor photodiode 144 and wakes up the controller 134 via the 
1 0 power manager 151 on a pen-down event. 

The radio transceiver communicates in the unlicensed 900MHz band normally 
used by cordless telephones, or alternatively in the unlicensed 2.4GHz industrial, 
scientific and medical (ISM) band, and uses frequency hopping and collision detection to 
provide interference-free communication. 

15 6.3 Pen Optics 

As discussed above, the pen 101 optics is implemented by a moulded optics 

body 135. The optics that is implemented by the optics body 135 is illustrated 
schematically in Figure 48. The optics comprises a first lens 157 for focussing radiation 
from the infrared LED 131, a mirror 158, a beam splitter 159, an objective lens 160 and 
20 a second lens 161 for focusing an image onto image sensor 132. Axial rays 162 illustrate 
the optical path. 

The optical path is designed to deliver a sharp image to the image sensor 132 of 
that part 193 of the imaged surface which intersects the field of view cone 192, within 
required tilt ranges (see later). The primary focussing element is the objective lens 160. 

25 This is also used in reverse to project illumination from the IR illumination LED 131 
onto the surface ^yithin the field of view. Since it is impractical to place both the image 
sensor 132 and the IR LED 131 at the focus of the objective, a beam splitter 159 is used 
to split the path and separate relay lenses 157 and 161 in each path provides refocussing 
at the image sensor 13*2 and the IR LED 131 respectively. This also allows different 

30 apertures to be imposed on the two paths. 
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The edges of the image sensor 132 act as the field stop for the capture field, and 
the capture path is designed so that the resulting object space angular field of view is as 
required (i.e. just under 20° for the application of this embodiment). The illumination 
path is designed to produce the same object space field of view as the capture path, so 
5 that the illumination fills the object space field of view with maximum power and 
uniformity. 

The IR LED 131 is strobed in synchrony with frame capture. The use of 
focussed illumination allows both a short exposure time and a small aperture. The short 
exposure time prevents motion blur, thus allowing position tag data capture during pen 
10 movement. The small aperture allows sufficient depth of field for the full range of 
surface depths induced by tilt. The capture path includes an explicit aperture stop 191 for 
this purpose. 

Because the image sensor 132 has a strong response throughout the visible and 
near infrared part of the spectrum, it is preceded by an infrared filter 163 in the capture 
15 path so that it captures a clean image of the tag data on the surface, free from 
interference from other graphics on the surface which may be printed using inks which 
are transparent in the near infrared. 

6.4 Pen Processing 

When the stylus nib 121 or ink cartridge nib 119 of the pen 101 is in contact 

20 with a surface, the pen 101 determines its position and orientation relative to the surface 
at 100 Hz to allow accurate handwriting recognition (see the article by Tappert, C, C Y 
Suen and T Wakahara, "The State of the Art in On-Line Hand Writing Recognition" 
IEEE Transactions on Patent Analysis and Machine Intelligence, Vol 12, number 8, 
August 1990, the disclosure of which is incorporated herein by cross-reference). The 

25 force sensor photodiode 144 is utilised to indicate relative threshold whether the pen is 
"up" or "down". The force may also be captured as a continuous value, as discussed 
above, to allow the full dynamics of a signature to be verified. 

The pen 101 determines the position and orientation of its nib 119, 121 on the 
surface by imaging, in the infrared spectrum, an area of the surface in the vicinity of the 
30 nib 119, 121. It decodes the nearest tag data and computes the position of the nib 1 19, 
121 relative to the location tag from the observed perspective distortion on the imaged 
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tag and the known geometry of the pen optics 135 (see later). Although the position 
resolution of the tag may be low, the adjusted position resolution is quite high, and easily 
exceeds the 200 dpi resolution required for accurate handwriting recognition (see above 
reference). 

5 Pen 101 actions relative to a surface are captured as a series of strokes. A 

stroke consists of a sequence of time-stamped pen positions on the surface, initiated by a 
pen-down event and completed by the subsequent pen-up event. A stroke is also tagged 
with the region ID of the surface whenever the region ID changes, i.e. just at the start of 
the stroke under normal circumstances. As discussed above, each location tag includes 
1 0 data indicative of its position on the surface and also region data indicative of the region 
of the surface within which the tag lies. 

Figure 49 is a diagram illustrating location tag and stroke processing in the pen 
101. When the pen 101 is in the pen-up state, the pen controller 134 continuously . 
monitors the force sensor photodiode 144 for a pen-down condition (step 164). While the 

15 pen is in a pen-down state, the pen controller 134 continuously captures 165, 166 and 
decodes 167 tag data from location tags from the surface, infers the pen 101 position and 
orientation relative to the surface, 168 and appends the position data to the current stroke 
data (including the tag data and other information such as force, if it is being 
continuously monitored). On a pen-up event the pen controller 134 encrypts 170 the 

20 stroke data and transmits 171 the stroke data via the RF chip 133 and antenna 1 12, to the 
computing system. Note that the pen samples the nib force 172 in order to determine 
whether the stroke has been completed 1 73 and also to determine whether a new stroke 
is being started 174. 

Assuming a reasonably fast 8 bit multiply (3 cycles), the processing algorithm 
25 (see later) uses about 80% of the processor's time when the pen is active. 

If the pen is out of range of a computing system to transmit to, then it buffers 
digital ink in its internal memory. It transmits any buffered digital ink when it is next 
within range of a computing system. When the pen's internal memory is full the pen 
ceases to capture digital ink and instead flashes its error LED whenever the user attempts 
30 to write with the pen 101. 

Table 4 lists the components of the raw digital ink transmitted from the pen 
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